Concern group also says tenants affected by the plan will need help relocating as they may be left with few affordable options.